Edit on GitHub

Tutorial

This tutorial shows the functionalities of Extremes.jl. They are illustrated by reproducing some of the results shown by Coles (2001) in An Introduction to Statistical Modeling of Extreme Values.

Before executing this tutorial, make sure to have installed the following packages:

  • Extremes.jl (of course)
  • DataFrames.jl (for using the DataFrame type)
  • Distributions.jl (for using probability distribution objects)
  • Gadfly.jl (for plotting)

and import them using the following command:

julia> using Extremes, DataFrames, Distributions, Gadfly

Model for stationary block maxima

Port Pirie example

This section concerns the annual maximum sea-levels recorded at Port Pirie, South Australia, from 1923 to 1987. This dataset were studied by Coles (2001) in Chapter 3.

Load the data

Loading the annual maximum sea-levels at Port Pirie:

data = load("portpirie")
first(data,5)

5 rows × 2 columns

YearSeaLevel
Int64Float64
119234.03
219243.83
319253.65
419263.88
519274.01

Plotting the data using the Gadfly package:

plot(data, x=:Year, y=:SeaLevel, Geom.line)
Year 1820 1840 1860 1880 1900 1920 1940 1960 1980 2000 2020 2040 2060 2080 2100 1840 1845 1850 1855 1860 1865 1870 1875 1880 1885 1890 1895 1900 1905 1910 1915 1920 1925 1930 1935 1940 1945 1950 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010 2015 2020 2025 2030 2035 2040 2045 2050 2055 2060 2065 2070 2075 2080 1800 1900 2000 2100 1840 1845 1850 1855 1860 1865 1870 1875 1880 1885 1890 1895 1900 1905 1910 1915 1920 1925 1930 1935 1940 1945 1950 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010 2015 2020 2025 2030 2035 2040 2045 2050 2055 2060 2065 2070 2075 2080 h,j,k,l,arrows,drag to pan i,o,+,-,scroll,shift-drag to zoom r,dbl-click to reset c for coordinates ? for help ? 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5 6.0 6.5 7.0 2.00 2.05 2.10 2.15 2.20 2.25 2.30 2.35 2.40 2.45 2.50 2.55 2.60 2.65 2.70 2.75 2.80 2.85 2.90 2.95 3.00 3.05 3.10 3.15 3.20 3.25 3.30 3.35 3.40 3.45 3.50 3.55 3.60 3.65 3.70 3.75 3.80 3.85 3.90 3.95 4.00 4.05 4.10 4.15 4.20 4.25 4.30 4.35 4.40 4.45 4.50 4.55 4.60 4.65 4.70 4.75 4.80 4.85 4.90 4.95 5.00 5.05 5.10 5.15 5.20 5.25 5.30 5.35 5.40 5.45 5.50 5.55 5.60 5.65 5.70 5.75 5.80 5.85 5.90 5.95 6.00 6.05 6.10 6.15 6.20 6.25 6.30 6.35 6.40 6.45 6.50 0 2 4 6 8 2.0 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 3.0 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 4.0 4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 4.9 5.0 5.1 5.2 5.3 5.4 5.5 5.6 5.7 5.8 5.9 6.0 6.1 6.2 6.3 6.4 6.5 SeaLevel

GEV parameters estimation

In this example, the Generalized Extreme Value (GEV) distribution is fitted by maximum likelihood to the annual maximum sea-levels at Port-Pirie.

The data have been loaded in a DataFrame. The function gevfit can be called directly using the dataframe as the first argument and the data column symbol as the second argument as follows:

julia> fm = gevfit(data, :SeaLevel)
MaximumLikelihoodEVA
model :
	BlockMaxima
	data :		Array{Float64,1}[65]
	location :	μ ~ 1
	logscale :	ϕ ~ 1
	shape :		ξ ~ 1

θ̂  :	[3.874750223091266, -1.6192723640210762, -0.05010719929448139]

The function gevfit returns a MaximumLikelihoodEVA object which contains:

  • the structure name indicating in particular the estimation method (maximum likelihood in this example);
  • the statistical model (the stationary block maxima model in this example);
  • the location, log-scale and shape parameter estimates respectively in the vector $ θ̂ $.
Note

The function returns the estimates of the log-scale parameter $\phi = \log \sigma$.

Diagnostics plots

TODO

Return level estimation

T-year return level estimate can be obtained using the function returnlevel on a fittedEVA object. The first argument is the fitted model, the second is the return period in years and the last one is the confidence level for computing the confidence interval.

For example, the 100-year return level for the Port Pirie blockmaxima model and the corresponding 95% confidence interval can be obtained with this commands:

julia> r = returnlevel(fm, 100, .95)
ReturnLevel(MaximumLikelihoodEVA
model :
	BlockMaxima
	data :		Array{Float64,1}[65]
	location :	μ ~ 1
	logscale :	ϕ ~ 1
	shape :		ξ ~ 1

θ̂  :	[3.874750223091266, -1.6192723640210762, -0.05010719929448139]
, 100, [4.688403360432851], [[4.377121171613511, 4.999685549252191]])

where the value can be accessed with

julia> r.value
1-element Array{Float64,1}:
 4.688403360432851

and where the corresponding confidence interval can be accessed with

julia> r.cint
1-element Array{Array{Float64,1},1}:
 [4.377121171613511, 4.999685549252191]
Note

In this example of a stationary model, the function returns a unit dimension vector for the return level and a vector containing only one vector for the confidence interval. The reason is that the function always returns the same type in the stationary and non-stationary case. The function is therefore type-stable allowing better performance of code execution.

To get the scalar return level in the stationary case, the following command can be used:

julia> r.value[]
4.688403360432851

To get the scalar confidence interval in the stationary case, the following command can be used:

julia> r.cint[]
2-element Array{Float64,1}:
 4.377121171613511
 4.999685549252191

Probability weighted moments estimation

Probability weighted moments estimation of the GEV parameters can also be performed by using the gevfitpwm function. All the methods also apply to the pwmEVA object.

julia> fm = gevfitpwm(data[:,:SeaLevel])
pwmEVA
model :
	BlockMaxima
	data :		Array{Float64,1}[65]
	location :	μ ~ 1
	logscale :	ϕ ~ 1
	shape :		ξ ~ 1

θ̂  :	[3.8731723562720766, -1.5932320395836068, -0.051477125862911276]

Bayesian estimation

Bayesian estimation of the GEV parameters can also be performed by using the gevfitbayes function. All the methods also apply to the BayesianEVA object.

julia> fm = gevfitbayes(data[:,:SeaLevel])

Progress:   0%|                                         |  ETA: 0:20:18
Progress:  12%|████▊                                    |  ETA: 0:00:04
Progress:  18%|███████▍                                 |  ETA: 0:00:03
Progress:  23%|█████████▌                               |  ETA: 0:00:02
Progress:  28%|███████████▍                             |  ETA: 0:00:02
Progress:  34%|██████████████▏                          |  ETA: 0:00:02
Progress:  40%|████████████████▍                        |  ETA: 0:00:02
Progress:  47%|███████████████████▏                     |  ETA: 0:00:01
Progress:  53%|█████████████████████▋                   |  ETA: 0:00:01
Progress:  58%|███████████████████████▋                 |  ETA: 0:00:01
Progress:  64%|██████████████████████████▍              |  ETA: 0:00:01
Progress:  70%|████████████████████████████▊            |  ETA: 0:00:01
Progress:  77%|███████████████████████████████▌         |  ETA: 0:00:01
Progress:  83%|█████████████████████████████████▉       |  ETA: 0:00:00
Progress:  88%|███████████████████████████████████▉     |  ETA: 0:00:00
Progress:  94%|██████████████████████████████████████▊  |  ETA: 0:00:00
Progress: 100%|█████████████████████████████████████████| Time: 0:00:02
BayesianEVA
model :
	BlockMaxima
	data :		Array{Float64,1}[65]
	location :	μ ~ 1
	logscale :	ϕ ~ 1
	shape :		ξ ~ 1

sim :	Mamba.Chains

Model for stationary threshold exceedances

The data of this section come from Chapter 4 of Coles (2001) and correspond to the daily rainfall accumulations at a location in south-west England from 1914 to 1962.

Load the data

Loading the daily rainfall at a location in South-England:

data = load("rain")
x = collect(Date(1914,1,1):Day(1):Date(1961,12,30))
data[!,:Date] = x
select!(data, [:Date, :Rainfall])
first(data,5)

5 rows × 2 columns

DateRainfall
Date…Float64
11914-01-010.0
21914-01-022.3
31914-01-031.3
41914-01-046.9
51914-01-054.6

Plotting the data using the Gadfly package:

plot(data, x=:Date, y=:Rainfall, Geom.point, Theme(discrete_highlight_color=c->nothing))
Date Jan 1, 1840 1850 1860 1870 1880 1890 1900 1910 1920 1930 1940 1950 1960 1970 1980 1990 2000 2010 2020 2030 2040 Jan 1, 1800 1850 1900 1950 2000 2050 Jan 1, 1800 1850 1900 1950 2000 2050 Jan 1, 1800 1850 1900 1950 2000 2050 h,j,k,l,arrows,drag to pan i,o,+,-,scroll,shift-drag to zoom r,dbl-click to reset c for coordinates ? for help ? -125 -100 -75 -50 -25 0 25 50 75 100 125 150 175 200 225 -100 -95 -90 -85 -80 -75 -70 -65 -60 -55 -50 -45 -40 -35 -30 -25 -20 -15 -10 -5 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 105 110 115 120 125 130 135 140 145 150 155 160 165 170 175 180 185 190 195 200 -100 0 100 200 -100 -90 -80 -70 -60 -50 -40 -30 -20 -10 0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 160 170 180 190 200 Rainfall

Threshold selection

TODO

GPD parameters estimation

Let's first identify the threshold exceedances:

threshold = 30.0
df = filter(row -> row.Rainfall > threshold, data)
first(df, 5)

5 rows × 2 columns

DateRainfall
Date…Float64
11914-02-0731.8
21914-03-0832.5
31914-12-1731.8
41914-12-3044.5
51915-02-1330.5

Get the exceedances above the threshold:

df[!,:Rainfall] =  df[!,:Rainfall] .- threshold
rename!(df, :Rainfall => :Exceedance)
first(df, 5)

5 rows × 2 columns

DateExceedance
Date…Float64
11914-02-071.8
21914-03-082.5
31914-12-171.8
41914-12-3014.5
51915-02-130.5

Generalized Pareto parameter estimation by maximum likelihood:

julia> fm = gpfit(df, :Exceedance)
MaximumLikelihoodEVA
model :
	ThresholdExceedance
	data :		Array{Float64,1}[152]
	logscale :	ϕ ~ 1
	shape :		ξ ~ 1

θ̂  :	[2.006896498380506, 0.1844926991237574]
Note

The function returns the estimates of the log-scale parameter $\phi = \log \sigma$.

Return level estimation

With the ThresholdExceedance structure, the returnlevel function requires several arguments to calculate the T-year return level:

  • the threshold value;
  • the number of total observation (below and above the threshold);
  • the number of observations per year;
  • the return period T;
  • the confidence level for computing the confidence interval.

The function uses the Peaks-Over-Threshold model definition (Coles, 2001, Chapter 4) for computing the T-year return level.

For the rainfall example, the 100-year return level can be estimated as follows:

julia> r = returnlevel(fm, threshold, size(data,1), 365, 100, .95)
ReturnLevel(MaximumLikelihoodEVA
model :
	ThresholdExceedance
	data :		Array{Float64,1}[152]
	logscale :	ϕ ~ 1
	shape :		ξ ~ 1

θ̂  :	[2.006896498380506, 0.1844926991237574]
, 100, [106.32558691303024], [[65.48163774428642, 147.16953608177405]])

where the value can be accessed with

julia> r.value
1-element Array{Float64,1}:
 106.32558691303024

and where the corresponding confidence interval can be accessed with

julia> r.cint
1-element Array{Array{Float64,1},1}:
 [65.48163774428642, 147.16953608177405]
Note

In this example of a stationary model, the function returns a unit dimension vector for the return level and a vector containing only one vector for the confidence interval. The reason is that the function always returns the same type in the stationary and non-stationary case. The function is therefore type-stable allowing better performance of code execution.

To get the scalar return level in the stationary case, the following command can be used:

julia> r.value[]
106.32558691303024

To get the scalar confidence interval in the stationary case, the following command can be used:

julia> r.cint[]
2-element Array{Float64,1}:
  65.48163774428642
 147.16953608177405

Probability weighted moments estimation

Probability weighted moments estimation of the GEV parameters can also be performed by using the gevfitpwm function. All the methods also apply to the pwmEVA object.

julia> fm = gpfitpwm(df, :Exceedance)
ERROR: MethodError: no method matching gpfitpwm(::DataFrame, ::Symbol)

Bayesian estimation

Bayesian estimation of the GEV parameters can also be performed by using the gevfitbayes function. All the methods also apply to the `BayesianEVA object.

julia> fm = gpfitbayes(df, :Exceedance)

Progress:  11%|████▍                                    |  ETA: 0:00:01
Progress:  22%|████████▉                                |  ETA: 0:00:01
Progress:  32%|█████████████▏                           |  ETA: 0:00:01
Progress:  44%|██████████████████▏                      |  ETA: 0:00:01
Progress:  55%|██████████████████████▌                  |  ETA: 0:00:00
Progress:  67%|███████████████████████████▍             |  ETA: 0:00:00
Progress:  78%|████████████████████████████████         |  ETA: 0:00:00
Progress:  90%|████████████████████████████████████▊    |  ETA: 0:00:00
Progress: 100%|█████████████████████████████████████████| Time: 0:00:00
BayesianEVA
model :
	ThresholdExceedance
	data :		Array{Float64,1}[152]
	logscale :	ϕ ~ 1
	shape :		ξ ~ 1

sim :	Mamba.Chains

Model for dependent data

Coles(2001, Chapter 5)

Model for non-stationary data

Coles(2001, Chapter 6)